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How Nebraska Teachers Use and Perceive Summative, 
Interim, and Formative Data 


Stephanie B. Wilkerson, Mary Klute, Beth Peery, and Jing Liu January 2021 


Teachers have access to more data than ever before, including summative (state-level), interim 
(benchmark-level), and formative (classroom-level) data. Yet research on how often and why teachers 
use each type of data is scarce. The Nebraska Department of Education partnered with the Regional 
| XolU Verh ule) ay-] MW] okele-] ho) a’ mG -Vaide-] maomeolale[UCoime-MiaU(oh mo) ma-t-lonl-)ecm-lole mm ola latell ol-] Mla MesoSoM) [=] o) e-1 ¢- mya store) [3 
to learn about teachers’ use and perceptions of summative, interim, and formative data and inform a 
state-level professional learning plan to support teachers’ data use. The results indicated that 29 percent 
of teachers did not use summative data, 21 percent did not use interim data, and 6 percent did not use 


formative data. Among teachers who reported using each type of data, teachers used formative data 
almost weekly and interim data about monthly to tailor instruction, identify instructional content, 
recommend additional student support, and group students. Teachers used summative, interim, and 
formative data least often in discussions about student learning with principals, colleagues, parents, 
and students. Teachers with 12 or fewer years of experience in education reported using formative data 
more often than did teachers with 22 or more years of experience. Teachers’ perceived competence in 
using data, their attitudes toward data, and their perceptions of organizational supports for using data 
(professional learning, principal leadership, and computer systems) were each positively associated with 
their use of data to inform instruction. When teachers reported greater perceived competence in using 
data, more positive attitudes toward data, or more organizational supports for using data, they more 
often used formative and interim data to inform instruction. Teachers with a more-advanced degree 
(education specialist or doctoral degree) reported feeling more competent in using data and having more 
positive attitudes toward data than did teachers with a bachelor’s degree. 


Since the start of the standards-based reform movement, state education agencies have developed accountabil- 
ity systems that use state-level, standardized summative assessments to measure student achievement against 
state content standards. However, policymakers, educators, and researchers have increasingly recognized the 
limitations and unintended consequences of using a state-level test as the sole data source for accountability 
purposes (Burger & Krueger, 2003; Hofman et al., 2015). For example, basing school accountability classifications 
only on state-level, standardized summative assessments might create a high-stakes environment in which teach- 
ers narrow the curriculum to only what the assessment covers (“teach to the test”) and prioritize instructional 
support for students who are close to passing the assessment and are most likely to affect an accountability clas- 
sification (Booher-Jennings, 2005; Datnow & Park, 2018). 


Under the Every Student Succeeds Act of 2015, the Nebraska Department of Edu- 
cation was one of several state education agencies to shift from using a single, 
summative measure of student learning to using multiple assessment measures For additional information, 
in an effort to form a more complete and accurate picture of student learning. including background 

This comprehensive approach typically consists of multiple, mutually supportive on the study, technical 
assessments that provide different types and levels of data—summative (state 
level), interim (benchmark level), and formative (classroom level)—to inform 
state policy, programs, and classroom instruction (see box 1 for definitions of key 
terms; Chappuis et al., 2009; Marsh et al., 2006). 


methods, and supporting 
analyses, access the 
report appendixes at 
https://go.usa.gov/xAXnM. 
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Box 1. Key terms 


Actions. Actions are how teachers use data to inform instructional practice, referred to frequently in this report as “instructional 
actions.” Teachers may take a variety of actions inside and outside the classroom based on assessment data, such as tailoring 
instruction to individual students’ needs, identifying students who need further assistance, selecting instructional materials, orga- 
nizing students in instructional groups, discussing data with students or parents, and meeting with other teachers about data 
(Datnow et al., 2007; Hamilton et al., 2009; Supovitz, 2012). 


Formative assessment. The Nebraska Department of Education (2018b, p. 2) defines formative assessment as “[f]ormal and 
informal tools teachers use in the classroom to check their students’ understanding and then adapt their teaching in the moment 
to what students need.” 

Types of formative assessments and teacher practices with formative data vary widely (Bennett, 2011). Generally, formative 
assessment is a process using formal and informal tools, as opposed to a specific test, that teachers implement during instruction 
to gather and interpret data about student learning and then use those data “in the moment” to direct classroom teaching and 
further student learning. Assessment typically occurs frequently during short cycles ranging from moment by moment to within 
and between lessons (Klute et al., 2017; Perie et al., 2007). Examples of formative assessments include teacher-developed quizzes, 
interactive class discussions, on-the-spot checks, and exit tickets (students write down one thing they learned on a note card and 
hand it to the teacher when they leave the classroom), among many other types of real-time assessments that provide teachers 
with immediate feedback. Information from a short-cycle formative assessment process can address a question such as “What 
instruction can | provide to help this student learn now?” (Gong, 2010). 


Interim assessment. The Nebraska Department of Education (2018b, p. 2) defines interim assessment as “[a]ssessments admin- 
istered at intervals between instruction (typically fall/winter/spring) to help teachers better understand student learning needs 
and determine growth toward learning targets.” The Nebraska Department of Education uses MAP Growth, created by the North- 
western Evaluation Association (NWEA), as the state’s interim assessment. This report uses “interim data” to refer to data derived 
from the NWEA MAP Growth interim assessment. 

Interim assessments are designed to inform classroom-level and district-level decisions about student learning relative to a 
defined set of academic goals or benchmarks (Farley-Ripple et al., 2020). Administration typically occurs within a medium-cycle 
timeframe (for example, at the beginning, middle, and end of the school year) and is likely directed by the school or district (Perie 
et al., 2007). Results can be aggregated and reported across classrooms or at the school, district, or state level, which allows for 
comparing results and tracking progress across these levels (Farley-Ripple et al., 2020). Interim assessments tend to be more 
formal than formative assessments and can include assessments created by states, districts, or assessment publishers. Informa- 
tion from periodically administered interim assessments can address questions such as “What effect has a program or interven- 
tion had on student learning?” “What are areas of strengths and deficits of students at the beginning, middle, and end of a school 
year?” and “Are students on track to meet end-of-year benchmarks?” (Gong, 2010). 


Nebraska school accountability classifications. By considering multiple indicators of school performance, the Nebraska Depart- 
ment of Education (2018a) classifies each Nebraska school into one of four levels: excellent, great, good, and needs improvement. 
The indicators include statewide math and English language arts assessment scores and trends, graduation rates, student absen- 
teeism rates, and English learner students’ progress toward proficiency. The Nebraska Department of Education uses this classifi- 
cation system to strategically target resources and support to schools most in need of improvement (see appendix A). 


Summative assessment. The Nebraska Department of Education (2018b, p. 2) defines summative assessment as “[c]ulminating 
assessments measuring student performance against state content area standards.” The Nebraska Student-Centered Assessment 
System (NSCAS) summative assessments are criterion-referenced, meaning that they measure student performance against a fixed 
set of criteria for the state content area standards. The NSCAS includes the following summative assessments: NSCAS English lan- 
guage arts, math, and science assessments for grades 3-8; the ACT for grade 11; and the English Language Proficiency Assessment 
for the 21st Century for English learner students. Teachers administer these summative assessments once a year. This report uses 
“summative data” to refer exclusively to data derived from the NSCAS summative assessments and not from other classroom-based 
measures teachers might use summatively, such as chapter or unit tests, research projects, or performance portfolios. 

State-level standardized summative assessments are typically used as an accountability measure of student performance 
compared with defined content standards or a group of students, such as a national or state sample of students in the same grade 
(Perie et al., 2007). Information from state-level standardized summative assessments can address questions such as “How do 
students in this school perform on state tests?” “What does student performance look like at the end of the year?” and “How is 
student performance changing over time?” (Gong, 2010). 
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Through a multiple-assessment approach, teachers can access summative, interim, and formative data, each of 
which provide different information that can inform different kinds of instructional decisions (Marsh et al., 2006). 
For example, state-level, standardized summative assessment data, collected once a year, can provide informa- 
tion about students’ performance at the end of the year and changes in students’ achievement over time related 
to state learning standards (Gong, 2010). Interim assessment data, collected at intervals during the school year, 
can help teachers monitor students’ rates of growth toward state, district, and school benchmarks and adjust 
instructional plans, program administration, and resource allocation to meet the unique needs of individual stu- 
dents (Chappuis & Stiggins, 2008; Hamilton et al., 2009; Perie et al., 2007). Formative data, collected “in the 
moment” during instruction, can help teachers plan which materials to use in class, group students for instruc- 
tion, tailor instruction for individual students, and identify additional instructional supports (Datnow et al., 2007; 
Lachat & Smith, 2005; Supovitz, 2012). 


The Nebraska Department of Education (2018b) developed a multiple-assessment approach, the Nebraska Stu- 
dent-Centered Assessment System (NSCAS), which identifies summative, interim, and formative assessments as 
measures of student learning. These include state-level, standardized summative assessments for English lan- 
guage arts, math, science, college readiness, and English language proficiency; Northwestern Evaluation Asso- 
ciation’s MAP Growth interim assessment; and classroom-based formative assessments. The NSCAS is intended 
to support teachers in using multiple assessments to measure what students know and need to learn and adjust 
instruction to help students stay on track with state content standards. To this end, supporting teachers’ data use 
by providing a clearly defined vision and plan for professional learning opportunities is a key component of the 
NSCAS. 


Given the Nebraska Department of Education’s investment in the NSCAS, department leaders were committed 
to developing a state-level professional learning plan to support its use. The leaders partnered with the Regional 
Educational Laboratory Central to conduct a study of how teachers used and perceived three types of data— 
summative, interim, and formative—that were at the core of Nebraska’s multiple-assessment approach. The study 
employed the Teacher Data Use Survey (Wayman et al., 2016) to measure how often teachers used data from the 
NSCAS standardized summative assessments, the MAP Growth interim assessment, and classroom-based forma- 
tive assessments; how they used these data to guide instruction; what their general perceptions and attitudes 
about data were; and how those perceptions compared with principals’ perceptions of teachers’ data use. Depart- 
ment leaders were also interested in learning whether teachers’ data use varied by teacher characteristics such 
as highest degree earned and years of experience in education, or by school characteristics, including Nebraska 
school accountability classifications, which are based on school performance indicators (see boxes 1 and 2 for 
more information on teacher and school characteristics and see appendix A for a summary of the literature on 
teachers’ data use and teacher and school characteristics). 


Understanding how teachers use and perceive data can help Nebraska education leaders (department staff and 
professional learning service providers) develop a data-informed vision and plan for state-level professional 
learning to support teachers’ data use. The results can help Nebraska education leaders identify areas in which 
teachers might need more support, such as understanding the purposes of each type of assessment, interpreting 
data from each assessment, and using data to inform instruction. The findings can also help Nebraska education 
leaders determine whether and how to differentiate their professional learning support based on, for example, 
school accountability classification or teachers’ highest degree earned or years of experience. 
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Research questions 


This study addressed five research questions to examine Nebraska teachers’ use of summative, interim, and for- 
mative data as well as their perceptions of and attitudes toward data: 


1. How do Nebraska teachers report using summative, interim, and formative data? 


2. Are Nebraska principals’ attitudes about data and perceptions of teachers’ data use similar to teachers’ reports 
of their own attitudes and data use? 


3. How does teachers’ use of data relate to teachers’ perceived competence in using data, attitudes toward data, 
and perceptions of organizational supports for using data? 


4. How do teachers’ use of data, perceived competence in using data, attitudes toward data, and perceptions of 
organizational supports for using data vary by teacher characteristics? 


5. How do teachers’ use of data, perceived competence in using data, attitudes toward data, and perceptions 
of organizational supports for using data vary based on Nebraska school accountability classifications (that is, 
excellent, great, good, and needs improvement) for the 2018/19 school year? 


For information on sources, sample, and methods used to address the research questions, see box 2 and appendix B. 


Box 2. Data sources, sample, and methods 


Data sources. This study used two sources of data: administrative data from the Nebraska Department of Education and survey 
data from the Teacher Data Use Survey (TDUS; Wayman et al., 2016). 

Administrative data included school-level and teacher-level variables. School-level variables included school level (elemen- 
tary, middle, or high); accountability classification (excellent, great, good, or needs improvement); and Title | status (schools that 
receive federal funds under Title |, Part A, of the Every Student Succeeds Act; U.S. Department of Education, 2018). Teacher-level 
variables included: 

e Highest degree earned: bachelor’s degree (or less’), master’s degree, or more-advanced degree (an education specialist or 
doctoral degree). 

¢ Special education endorsement: whether a teacher held a special education endorsement. 

e¢ Core subject teacher: whether a teacher taught subjects such as general elementary, English language arts, math, science, or 
history/social studies at the time of survey administration. Teachers were considered noncore subject teachers if they taught 
any other subject, such as fine arts, health/physical education, or a foreign language, at the time of survey administration. 

e Years of experience in education: number of years in education. For analyses, years of experience were converted into quar- 
tiles: 5 or fewer years, 6-12 years, 13-21 years, or 22 or more years. 

Survey data included teachers’ and principals’ responses to the TDUS (see appendixes C and D). The TDUS measured teachers’ 
actions with summative, interim, and formative data; their perceived competence in using data; their attitudes toward data; and 
their perceptions of organizational supports for using data (see appendix E). The principal version of the survey included similarly 
worded items that measured principals’ perceptions of teachers’ data use (see appendix E). Each scale is described below. 

The Actions with Data scale in the TDUS measured the frequency of specific actions that teachers reported taking with sum- 
mative, interim, and formative assessment data. These actions included identifying instructional content to use in classes, tailor- 
ing instruction to individual students’ needs, discussing data with students or parents, and meeting with other teachers about 
data.” The survey focused on the assessments used in Nebraska: the Nebraska Student-Centered Assessment System summative 
assessments and MAP Growth interim assessments. The survey did not reference a specific formative assessment because forma- 
tive assessments include a variety of formal and informal tools.? Respondents rated items on a four-point frequency scale. 

The Competence in Using Data scale in the TDUS included items that prompted respondents to consider whether they were 
proficient in various aspects of data use. When answering these items, respondents were prompted to think more generally about 
their data use to inform education practice rather than about a specific type of data (that is, summative, interim, or formative). 
Example items included “I am good at using data to diagnose student learning needs” and “I am good at adjusting instruction 
based on data.” Respondents rated items on a four-point agreement scale. 
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The Attitudes toward Data scale in the TDUS included two subscales: Attitudes toward Data and Data’s Effectiveness for Peda- 
gogy. The Attitudes toward Data subscale included items such as “I think it is important to use data to inform education practice” 
and “I like to use data.” The Data’s Effectiveness for Pedagogy subscale included items such as “Data help teachers plan instruc- 
tion” and “Students benefit when teacher instruction is informed by data.” Respondents rated items on a four-point agreement 
scale. When answering these items, respondents were prompted to think more generally about using data to inform education 
practice rather than about a specific type of data (that is, summative, interim, or formative). 

The Organizational Supports scale in the TDUS included three subscales: Computer Data Systems, Support for Data Use, and 
Principal Leadership. The Computer Data Systems subscale included items about various characteristics of respondents’ data 
systems, such as “I have the proper technology to efficiently examine data.” The Support for Data Use subscale included items 
about various support structures for teachers, such as “Il am adequately supported in the effective use of data.” Finally, the Princi- 
pal Leadership subscale included items about actions that principals and assistant principals take with data, such as “My principal 
or assistant principal(s) encourages data use as a tool to support effective teaching.” Respondents rated items on a four-point 
agreement scale. When answering these items, respondents were prompted to think more generally about using data to inform 
their education practice rather than about a specific type of data (that is, summative, interim, or formative). 


Sample. The study team examined survey responses from 3,572 teachers and 171 principals across 353 schools. These respon- 
dents represented 35 percent of the teachers and 48 percent of the principals who were sent the survey. The study team applied 
nonresponse weights to generate findings that were likely representative of the entire group of educators originally surveyed (see 
appendix B for details on this process and characteristics of the analytic samples). 


Survey administration. The Nebraska Department of Education (NDE) sent invitations to complete the online survey to all teach- 
ers and principals in selected schools in March 2019. Survey recipients received three follow-up reminders. To address common 
challenges with self-reported data, NDE sent recipients a letter from the Nebraska commissioner of education stating that the 
survey results would be used to plan ongoing professional learning support for data use and would not be used for accountability 
purposes. NDE also assured recipients that their identities and confidentiality would be protected and that principals would not 
receive individual teachers’ survey results or know which teachers completed the survey. To promote memory recall and minimize 
the time between assessment and survey completion, NDE administered the survey at a time during the 2018/19 school year when 
teachers had had the opportunity to use summative, interim, and formative data. 


Analyses. To address the first two research questions, the study team calculated descriptive statistics, including scale means and 
standard deviations of survey responses from teachers and principals. The study team followed the TDUS guide (Wayman et al., 
2016), which suggests that using scale means is appropriate when making comparisons across survey scales and survey versions. 
Mean differences of 0.25 or larger are presented in this report. For example, on a four-point scale in which 1 indicates strongly 
disagree, 2 indicates disagree, 3 indicates agree, and 4 indicates strongly agree, the difference between a teacher mean rating of 
2.50 and a principal mean rating of 2.75 would meet the threshold for reporting. 

To address the remaining research questions, the study team used multilevel regression models. Multilevel models account 
for teachers being clustered within schools. It is important to account for this because teachers in the same schools are likely to be 
more similar to one another than to teachers in other schools. The models controlled for school characteristics that could relate to 
teachers’ data use, including school level (elementary, middle, or high) and Title | status. The models also controlled for teachers’ 
background characteristics, including highest degree earned, years of experience in education, special education endorsement, 
and teaching assignment in a core subject. For research question 3, which examined associations between teachers’ actions with 
data and teachers’ perceptions of and attitudes toward data, the report includes only associations that were significant at p < 
.01 or better. For research questions 4 and 5 the report presents regression-adjusted means to describe differences between 
groups of teachers with particular characteristics. All statistically significant differences of 0.25 or larger between subgroups are 
described in the report. 

The report describes teachers’ data use and associations between school characteristics and teachers’ use of, attitudes 
toward, and perceptions of data. The analyses do not support any causal claims. Therefore, no inferences are made about the 
effects of teacher or school characteristics on teachers’ use of, attitudes toward, or perceptions of data. 


Notes 

1. Only 0.3 percent of the sample had less than a bachelor’s degree (for example, a certificate). 

2. There might be other actions that teachers take with summative, interim, or formative data but that this study did not measure. 

3. A possible study limitation is that because the survey did not provide a definition of formative assessments, teachers might have responded to survey 
questions based on different interpretations of what constitutes a formative assessment. The Nebraska Department of Education provides teachers 


with a definition of formative assessments (see box 1), but teachers might or might not have had this definition in mind when completing the survey. 
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Findings 


This section presents findings for each of the five research questions. Supporting analyses are in appendix C, and 
additional analyses of Teacher Data Use Survey data are in appendix D. 


Not all Nebraska teachers used summative, interim, and formative data to inform instruction, but 
teachers who did use data used formative data more often than summative and interim data 


About 29 percent of teachers reported that they did not use summative data, 21 percent reported that they did 
not use interim data, and 6 percent reported that they did not use formative data. 


Among teachers who reported using each type of data, use of summative, interim, and formative data was con- 
sistent with how often each assessment is typically administered. That is, teachers used formative data from 
ongoing classroom-based formative assessments most often and used summative data from the annual state 
assessment least often. However, teachers reported using summative and interim data with a frequency that 
surpassed how often each assessment is typically administered. For example, teachers administer state summa- 
tive assessments once a year, whereas the mean for teachers’ use of summative data (1.63) was about once a 
month (figure 1). For interim assessments, typically administered three times a year, the mean for teachers’ use 
of interim data (1.98) was about once or twice a month. For formative assessments, which teachers generally 
administer on an ongoing basis, the mean for teachers’ use of formative data (3.03) was weekly or almost weekly. 


Nebraska teachers used formative data almost weekly and interim data about monthly to inform a variety of 
instructional actions (figure 2). The means for teachers’ instructional actions with formative data corresponded 
to weekly or almost weekly for tailoring instruction to individual students’ needs (2.96), identifying instructional 
content (2.92), developing recommendations for additional instructional support (2.88), and grouping students 
for targeted instruction (2.69). In comparison, the means for teachers’ taking these instructional actions with 
interim data corresponded to monthly or almost monthly, and summative data less than once a month." 


Figure 1. Teachers used formative data more often than summative and interim data, 2019 


A few times 
a week (4) 


Weekly or almost 
weekly (3) 


Once or twice 


a month (2) 
Less than once ial 
eal) Summative data use Interim data use Formative data use 
(n = 2,268) (n = 2,643) (n = 3,016) 


Source: Authors’ analysis of 2019 data from the Teacher Data Use Survey administered by the Nebraska Department of Education. 


1. See table C5 in appendix C for item-level means of actions with summative, interim, and formative data on the same rating scale (that 
is, with the summative rating scale recoded to the rating scale for interim and formative data for ease of comparison). See table D6 in 
appendix D for the original means for actions with summative data. 
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Figure 2. Teachers took instructional actions with formative data more often than with interim and 
summative data, 2019 


m= Summative data m Interim data m Formative data 
(n = 2,268) (n = 2,643) (n = 3,016) 


To tailor instruction to individual students’ needs. 


To identify instructional content to use in class. 


To develop recommendations for 
additional instructional support. 


To form small groups of students 
for targeted instruction. 


To discuss data with a student. 


To meet with another teacher about data. 


To meet with a specialist (e.g., instructional 
coach or data coach) about data. 


To discuss data with a parent or guardian. 


PPE PPEE 


Less than once Monthly Weekly A few times 
a month (1) (2) (3) a week (4) 
Frequency of actions 


Note: Frequency of actions with summative data was measured on a scale in which 1 = one or two times a year, 2 = a few times a year, 3 = monthly, and 
4 = weekly. To make visual comparisons using the same survey scale, ratings for summative data were recoded to the interim and formative response 
scale. A rating of 4 (weekly) on the summative scale was recoded as a 3 (weekly or almost weekly) on the interim/formative scale, a rating of 3 (monthly) 
was recoded as a 2 (once or twice a month), and a rating of 2 or 1 was recoded as a1 (less than once a month). 


Source: Authors’ analysis of 2019 data from the Teacher Data Use Survey administered by the Nebraska Department of Education. 


Teachers used the three types of data least often to guide discussions with students, other teachers, instruction- 
al specialists, and parents or guardians. In particular, these findings suggest that teachers discussed data about 
student learning with parents or guardians less than monthly. 


Principals had more positive perceptions of and attitudes toward data than teachers did 
On average, principals and teachers differed in their ratings of the usefulness of summative and interim data, but 
they had similarly favorable ratings of the usefulness of formative data to inform teachers’ practice (figure 3). 


Both principals and teachers perceived summative data to be less useful than interim and formative data for 
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Figure 3. Teachers and principals had different perceptions of the usefulness of summative and interim data 
and similar perceptions of the usefulness of formative data, 2019 


| Teachers m® Principals 


Very useful (4) 


Useful (3) 


Somewhat useful (2) 


Not useful (1) 


Summative data Interim data Formative data 


Note: For teachers n = 3,216 for summative data, n = 3,340 for interim data, and n = 3,203 for formative data. For principals n = 171 for summative data, 
n= 168 for interim data, and n = 164 for formative data. 


Source: Authors’ analysis of 2019 data from the Teacher Data Use Survey administered by the Nebraska Department of Education. 


informing teacher practice. However, principals’ mean rating of summative data (2.35) was more positive than 
somewhat useful, and teachers’ mean rating (1.83) was less positive than somewhat useful. Principals’ percep- 
tions of interim data represented a mean rating of useful to very useful to inform teacher practice (3.42). On the 
other hand, teachers’ perceptions of interim data corresponded to a mean rating of somewhat useful to useful to 
inform practice (2.61). 


Principals and teachers provided similar ratings, on average, of their perceptions of teacher competence in using 
data, their attitudes toward data, and their perceptions of organizational supports for using data (figure 4). 


Figure 4. Teachers and principals were similar in their perceptions of teacher competence in using data and of 
organizational supports for using data, but differed in their attitudes toward data, 2019 


m Teachers &® Principals 
(n=3,572)  (n=171) 


Strongly agree (4) 


Agree (3) 


Disagree (2) 


Strongly disagree (1) ae 
Attitudes Perceptions of organizational Perceptions of teacher 
toward data supports for data use competence in using data 


Source: Authors’ analysis of 2019 data from the Teacher Data Use Survey administered by the Nebraska Department of Education. 
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Perceptions about teacher competence in using data were comparable among principals (mean of 2.86) and 
teachers (mean of 2.91). For the Competence in Using Data scale, both principals and teachers provided similar 
ratings on statements about teachers being good at using data to diagnose student learning needs, adjusting 
instruction based on data, planning lessons, and setting student learning goals (see table C9 in appendix C for 
item-level means). 


Although both principals and teachers reported positive attitudes toward data, principals’ attitudes toward teach- 
ers’ data use were more positive (mean of 3.43) than were teachers’ attitudes toward their own data use (mean of 
3.09). Specifically, on the Attitudes toward Data scale, principals’ attitudes toward data were more positive than 
were teachers’ attitudes on statements about liking to use data; believing that students benefit when instruction 
is informed by data; believing that using data helps them be better educators; and finding data useful, important 
to educational practice, and helpful in planning instruction (see table C7 in appendix C for item-level means). 


Ratings of organizational supports for using data were similar among principals (mean of 3.11) and teachers (mean of 
2.95), including perceptions of teachers’ access to professional learning support for using data, principals’ leadership 
for data use, and computer data systems (see figure 4). Principals’ and teachers’ perceptions of principals’ support 
for teachers’ data use and computer systems that support data use were comparable, although their ratings differed 
on four items. Specifically, principals’ ratings were more positive than teachers’ ratings on items about principals dis- 
cussing data with teachers, principals creating protected time for using data, teachers having the proper technology 
for using data, and computer systems providing access to data (see table C8 in appendix C for item-level means). 


Teachers’ self-reported use of data was positively associated with their perceived competence in 
using data, their attitudes toward data, and their perceptions of organizational supports for using 
data 


The associations between teachers’ reported data use and their perceived competence in using data, their atti- 
tudes toward data, and their perceptions of organizational supports for using data varied slightly by the type 
of data (summative, interim, or formative). Ratings on the Competence in Using Data and Organizational Sup- 
ports scales were statistically significantly and positively associated with teachers’ reported use of all three types 
of data (table 1). Teachers who reported greater perceived competence in using data or more organizational 


Table 1. Teachers’ perceived competence in using data and of organizational supports for using data were 
statistically significantly and positively associated with their reported use of summative, interim, and 
formative data, 2019 


Use of Use of Use of 
Perceptions about data scales summative data interim data formative data 
Competence in Using Data 0.19*** 0.23*** 0.30*** 
Attitudes toward Data 0.06 0.08 0.13*** 
Organizational Supports 0.20*** 0.22*** 0.14*** 


*** Significant at p< .001. 


Note: Coefficients are from two-level multiple regression models that accounted for the nesting of teachers within schools. Coefficients describe the 
amount of change that would be expected in the data use variables with a one-point increase in the perceptions variables. For example, teachers 

who reported that, on average, they agreed with the statements about competence in using data (a rating of 3) would be expected to have “use of 
summative data” ratings that were 0.19 point higher than the ratings by teachers who reported, on average, that they disagreed with the statements 
about competence in using data (a rating of 2). Separate models were fit for summative data (n = 2,277 teachers in 350 schools), interim data (n = 2,665 
teachers in 351 schools), and formative data (n = 3,013 teachers in 353 schools). Sample sizes varied across models because teachers were asked about 
their use of a particular type of data only if they reported having access to that type of data. All models controlled for the following variables: teach- 
ers’ highest degree earned, special education endorsement, core subject, and years of experience, as well as the school level and Title | status of their 
schools. See tables C10—C12 in appendix C for complete model results. 


Source: Authors’ analysis of 2019 data from the Teacher Data Use Survey administered by the Nebraska Department of Education. 
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supports for using data also reported taking more frequent instructional actions informed by summative, interim, 
and formative data. Teachers’ attitudes toward data were statistically significantly and positively associated with 
only their reported use of formative data. That is, when teachers reported having more positive attitudes toward 
data, they indicated that they more often used formative data to inform instruction. 


Teachers’ highest degree earned was positively associated with their perceived competence in using 
data and their attitudes toward data 


Teachers who had a more-advanced degree reported greater perceived competence in using data than did 
teachers who had a bachelor’s degree (figure 5). The difference was statistically significant. On average on the 
four-point Competence in Using Data scale, teachers with a more-advanced degree had a perceived competence 
rating of 3.17, and teachers with a bachelor’s degree had a rating of 2.88, where 2 indicated disagreement, 3 indi- 
cated agreement, and 4 indicated strong agreement with the statements on the scale. 


Teachers’ attitudes toward data followed a similar pattern. Teachers with a more-advanced degree reported more 
positive attitudes toward data than did teachers who had a bachelor’s degree (see figure 5). This difference was 
Statistically significant. The mean rating of attitudes toward data for teachers with a more-advanced degree was 
3.36. On average on the four-point Attitudes toward Data scale, teachers with a bachelor’s degree had a rating of 
3.07, where 3 indicated agreement and 4 indicated strong agreement with the statements on the scale. 


Teachers with 12 or fewer years of experience in education reported using formative data more often 
than did teachers with 22 or more years of experience 


Teachers with the least amount of experience, 5 or fewer years, reported using formative data most often, on 
average (2.65), followed by teachers with 6-12 years of experience (2.56; figure 6). The differences between each 
of these groups and teachers with the most experience, 22 or more years, were statistically significant. On a 


Figure 5. Teachers with a more-advanced degree reported greater perceived competence in using data and 
more-positive attitudes toward data than did teachers with a bachelor’s degree, 2019 


m Bachelor’s degree ™ Master’s degree ™ More-advanced degree® 


Strongly agree (4) 


Agree (3) 


Disagree (2) 


Strongly disagree (1) 


Perceptions of teacher Attitudes 
competence in using data toward data 


Note: Adjusted means were calculated from two-level multiple regression models that accounted for the nesting of teachers within school (n = 3,572 
teachers in 353 schools). All models controlled for the following variables: teachers’ highest degree earned, special education endorsement, core sub- 
ject, and years of experience, as well as the school level and Title | status of their schools. See tables C13—C16 in appendix C for complete model results. 


a. Includes education specialist degree and doctoral degree. 


Source: Authors’ analysis of 2019 data from the Teacher Data Use Survey administered by the Nebraska Department of Education. 
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Figure 6. Teachers with the least experience used formative data more often than did teachers with the most 
experience, 2019 


A few times a week (4) 


Weekly (3) 


2.37 
Monthly (2) — - : - = = L . = 
Less than once a month (1) 


5 or fewer years 6-12 years 13-21 years 22 years or more 
Years of experience 
Note: Adjusted means were calculated from a two-level multiple regression models that accounted for the nesting of teachers within schools (n = 3,013 
teachers in 353 schools). All models controlled for the following variables: teachers’ highest degree earned, special education endorsement, core 


subject, and years of experience, as well as the school level and Title | status of their schools. See tables C23 and C24 in appendix C for complete model 
results. 


Source: Authors’ analysis of 2019 data from the Teacher Data Use Survey administered by the Nebraska Department of Education. 


four-point scale measuring frequency, teachers with 22 or more years of experience reported using formative 
data the least often (2.28), where 2 indicated once or twice a month and 3 indicated weekly. Teachers’ responses 
for frequency of summative and interim data use were more similar across the years of experience categories. 
No differences of .25 or greater were observed between the years of experience categories for summative and 
interim data (see tables C19—C22 in appendix C). 


Teachers working in schools with different accountability classifications had similar survey responses 


Teachers’ responses for frequency of summative, interim, and formative data use; perceived competence in using 
data; attitudes toward data; and organizational supports were similar across school accountability classifications 
(excellent, great, good, or needs improvement; see tables C25—C36 in appendix C). 


Implications 


Leaders at the Nebraska Department of Education can consider the following implications as they develop profes- 
sional learning plans for building teachers’ capacity to use data from multiple, complementary assessments for 
instructional decisionmaking. 


Considering principals’ support and perceptions of teachers’ data use 


To help teachers use summative, interim, and formative data, state education leaders could encourage principals 
to provide the necessary organizational supports for effective data use in their schools (Gerzon & Guckenburg, 
2015; Schildkamp et al., 2019). Overall, Nebraska teachers in this study had positive perceptions about the orga- 
nizational supports available to them to use data, including support from principals. However, when considering 
the professional learning needs of teachers and principals, Nebraska education leaders might want to address the 
areas in which teachers’ perceptions of data use in general were not as positive as principals’ perceptions. These 
areas, such as principals creating protected time for using data and discussing data with teachers, could suggest 
opportunities for principals to convey to teachers the importance and value of using data to inform instruction. 
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Other findings, such as principals perceiving summative and interim data to be more useful than teachers per- 
ceive them to be, offer an opportunity to explore why their perceptions differ. Because this study found that 
teachers’ perceptions of and attitudes toward data were positively associated with the actions they reported 
taking with data, principals’ support for professional learning in these areas might be helpful in changing teach- 
ers’ instructional practice (Datnow & Hubbard, 2016; Reeves, 2017). 


Building capacity for data conversations and inquiry 


Nebraska education leaders might consider providing professional learning to support teachers in discussing data 
with colleagues and parents to address problems of practice and inform instructional decisionmaking. In exam- 
ining the actions teachers take with data, this study found that teachers reported using summative, interim, and 
formative data least often to support discussions with other teachers and specialists such as instructional coaches 
or data coaches, who typically have more experience in using data. They also engaged parents in discussions 
about student learning based on data less than monthly. In addition, teachers’ ratings were less positive than 
principals’ ratings about principals discussing data in general with teachers. 


These findings point to a possible opportunity to expand teachers’ data use by building their capacity to engage 
in collaborative data conversations with other teachers, specialists, principals, and parents aimed at collectively 
solving problems of practice. Engaging in data conversations can help teachers examine and understand data, 
shape how they perceive data, and aid them in determining what instructional actions to take (Beck & Nunnaley, 
2020; Bocala et al., 2014; Datnow & Hubbard, 2016). Resources such as Five Steps for Structuring Data-Informed 
Conversations and Action in Education (Kekahio & Baker, 2013) might help teachers, specialists, and principals 
engage more frequently and purposefully in data conversations that inform instruction and involve parents in 
understanding student learning. 


Understanding the process teachers use to translate data into instructional action, as well as the ways principals 
can support that process, might be an important area for Nebraska education leaders to explore further through 
research and professional learning support. Although this study measured how often teachers discussed data 
with others, it did not measure how those conversations might have informed instructional decisions or whether 
they occurred as part of a cycle of inquiry. When teachers engage in data-informed decisionmaking as part of a 
cycle of inquiry, they follow a systematic process for identifying a learning problem; setting a purpose or forming 
questions to address the problem; collecting, analyzing and interpreting data; taking action to change instruction; 
measuring whether the instructional changes worked; and then repeating the process (Kippers et al., 2018; Lipton 
& Wellman, 2012; Mandinach & Gummer, 2016b). 


Determining whether teachers’ perceived competence represents actual competence in using data 


By measuring teacher data competence, Nebraska education leaders could determine whether teachers’ per- 
ceived competence in using data reflects their actual proficiency based on the state’s well-defined data compe- 
tencies (Nebraska Department of Education, 2013). This study measured teachers’ perceived competence in using 
data, which was positive overall regardless of school accountability classification, but it did not measure teachers’ 
actual competence in using data. Nebraska education leaders might consider developing direct assessments of 
teachers’ actual competence in using data and comparing results to their perceived competence in using data. 
For example, assessment items could be added to existing district or state teacher evaluation rubrics so that 
administrators could use the rubrics to assess teachers’ data competence. Administrators could use information 
about teacher data competencies to develop a data-informed professional learning plan that focuses on areas in 
which Nebraska teachers need to improve the most. Education leaders in other states might consider how assess- 
ments of teachers’ perceived competence in using data and their actual competence in using data can inform 
state-level professional learning plans for their data use. 
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Education leaders in Nebraska might also consider strategies that could help close the gap in perceived compe- 
tence in using data between teachers who hold a more-advanced degree and those who hold a less-advanced 
degree (van Geel et al., 2017). This study found that teachers with a more-advanced degree reported greater per- 
ceived competence than did teachers with a bachelor’s degree. One possible strategy to close this gap is to create 
a statewide plan that articulates a coordinated system of support for building teachers’ competence in using data 
through preservice, in-service, and classroom-based professional learning support (Mandinach & Gummer, 2012; 
Mandinach & Jimerson, 2016). This multipronged approach could involve state credentialing and licensure agen- 
cies as well as teacher preparation programs to help ensure that teachers develop the knowledge and skills they 
need to use data effectively, regardless of education degree (Mandinach & Gummer, 2016a). 


Researching how teachers understand the intended purposes of using multiple assessments 


Further research could focus on how teachers understand the intended purposes of using summative, interim, 
and formative assessment data and how this understanding shapes their use and perceptions of the usefulness 
of those data. Previous research has shown that teachers’ understanding may influence how often they use 
each type of data and how they perceive the usefulness of the data (Farley-Ripple et al., 2020; Gong, 2010). For 
example, the current study found that teachers used classroom-based formative data more often and perceived 
formative data to be more useful than summative and interim data. It also found that 29 percent of teachers did 
not use summative assessment data and 21 percent did not use interim assessment data. In addition, because 
teachers typically develop their own formative assessments, researching the variation in how teachers use for- 
mative assessments could inform statewide guidance on what constitutes high-quality formative assessments. 
Research results also could inform how Nebraska education leaders communicate a state-level vision of the 
intended purposes for using data from summative, interim, and formative assessments as part of the Nebraska 
Student-Centered Assessment System. 


When developing professional learning plans for teachers’ data use, state leaders could help teachers elevate 
their use of summative, interim, and formative data for their intended purposes by offering training on how 
different types of data can inform different types of decisions. Although this study measured teachers’ use of 
summative, interim, and formative data, these are not the only types of data that teachers might use to inform 
instructional decisions. In addition, in professional learning plans, state education leaders might also address how 
teachers can use other types of data, such as behavioral data, student perceptual data, and student demographic 
data, for different purposes (Gummer & Mandinach, 2015; Hamilton et al., 2009). 
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